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ABSTRACT 

Genetic  Algorithm  (GA)  has  emerged  as  a  powerful 
tool  to  discover  optimal  for  multidimensional 
knapsack  problem  (MDKP).  Multidimensional 
knapsack  problem  has  recognized  as  NP-hard  problem 
whose  applications  in  many  areas  like  project 
selection,  capital  budgeting,  loading  problems,  cutting 
stock  etc.  Attempts  has  made  to  develop  cluster 
genetic  algorithm  (CGA)  by  mean  of  modified 
selection  and  modified  crossover  operators  of  GA. 
Clustered  genetic  algorithm  consist  of  (1)  fuzzy 
roulette  wheel  selection  for  individual  selection  to 
form  the  mating  pool  (2)  A  different  kind  of  crossover 
operator  which  employ  hierarchical  clustering  method 
to  form  two  clusters  from  individuals  of  mating  pool. 
CGA  performance  has  examined  against  GA  with 
respect  to  30  benchmark  problems  for  multi¬ 
dimensional  knapsack.  Experimental  results  show  that 
CGA  has  significant  improvement  over  GA  in  relation 
to  discover  optimal  and  CPU  running  time.  The  data 
set  for  MDKP  is  available  at 
http://people.brunel.ac.uk/ 
mastjjb/j  eb/ orlib/files/mknap2  .txt 

Keywords:  Multidimensional  knapsack  problem, 
Genetic  algo-rithm,  Fitness  function,  Crossover, 
Mutation 

1  INTRODUCTION 

Multidimensional  knapsack  problem  is  one  of  the 
famous  problem  in  applied  mathematics  belongs  to 
NP-  hard  combi-natorial  optimization  [1],  This 
problem  modeling  has  applied  in  several  project 
selection  and  capital  budgeting  areas  [2-4],  Meirier 
developed  more  mechanisms  from  real  word  practice 
to  combine  models  of  capital  budgeting  with  novels, 
and  practically  applicable  technique  for  evaluation  of 
project  [5].  They  developed  scenario  dependent 
capital  budgeting  model  that  have  MDKP  as  a 
subproblem  associated  with  constraint  of  generalized 
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upper  bound  (GUB).  Capital  budgeting  emerged  as 
key  challenge  for  not  for  profits  multihospital 
healthcare  units  in  United  States.  D.N.  Kleinmuntz 
and  C.E.  Kleinmuntz  developed  a  framework  for 
capital  budeting  which  used  MDKP  formulations  [6]. 
MDKP  have  been  employed  to  model  prob-lems  like 
investment  policy  in  tourism  area  of  developing 
country  [7],  database  allocation  and  processor 
allocation  in  distributed  computing  environments  [8], 
groceries  delivery  in  vehicles  with  multiple 
compartments  [9],  cutting  stock  [10],  loading 
problems  [11]  and  approval  voting  [12],  Currently, 
MDKP  has  been  applied  in  modeling  of  daily 
management  of  remote  satellite  like  SPOT,  which 
consisted  in  deciding  each  day  what  photographs  will 
be  attempted  the  next  day  [13]. 

Multidimensional  knapsack  problem  is  described  by  n 
objects  and  m  knapsacks.  Each  knapsack  has  a 
capacity  dj  (j  =  1;  2;  3:::m).  A  number  of  binary 
variables  x;  (i  =  1;  2;  3::n)  are  used,  that  is  set  to  1  if 
ith  object  is  chosen  to  inset  in  knapsacks  otherwise  it 
is  set  to  0.  Each  object  has  a  profit  value  pi  (i  =  1;  2; 
3::n)  and  a  weight  Wy  corresponding  to  knapsacks. 
Thus  mathematically  this  optimization  problem  can  be 
modeled  as  : 


n 

Xi 

Maximize 

=1 

(1) 

M 

& 

wjjxj  dj 

=i 

j  =  1:  2;  :::m 

(2) 

xi  2  fO;  lg 

i=  1:2;  3::n 

(3) 

The  goal  of  MDKP  is  to  insert  a  subset  of  objects  into 
knapsacks  that  obtain  maximum  profit  without 
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capacity  viola-tion  of  knapsacks.  This  characteristic 
reveal  this  problem  has  all  nonnegative  entries.  More 

specifically,  we  can  say  p;  >  0, 

m 

dj>0,0wjj  djand  wjj  djfor  all  i  2  n  and  j  2  m 

i=i 

Rest  of  the  paper  is  structured  as  follows.  Genetic 
algorithm  for  MDKP  is  described  in  Section  II. 
Section  III  give  the  overview  of  related  work  for 
MDKP  in  domain  of  genetic  algorithm.  Section  IV 
explain  CGA  to  solve  multidimensional  knapsack 
problem.  Experimental  environments  and  results  are 
given  in  Section  V.  At  last  conclusion  is  made  in 
section  VI. 


II.  GENETIC  ALGORITHM  FOR 

MULTIDIMENSIONAL  KNAPSACK 

PROBLEM 

Genetic  algorithm  is  computation  optimization 
technique  which  mimic  the  evolutionary  principle  i.e. 
fittest  individual  will  survive  in  each  generation.  GA 
is  guided  by  three  main  operation:  selection,  crossover 
and  mutation.  Individuals  re -production  are  governed 
by  crossover  and  mutation  operation.  Selection 
operation  choose  fittest  individuals  from  population  to 
form  mating  pool.  Algorithm  run  as  follow  :  A  set  of 
random  individuals  are  produced  which  serve  as  the 
initial  population  of  GA.  The  individual  can  be 
expressed  in  real  number,  binary  number,  integer 
number,  character  based  on  nature  of  problem.  MDKP 
uses  the  binary  representation.  Individual  goodness 
measurement  is  estimated  through  fitness  function. 
MDKP  fitness  function  is  maximize  the  objective 
function. 


Algorithm  1  Genetic  Algorithm  (GA) 


Initialize  the  control  parameters  ; 

Initialize  randomly  N  individuals  as  initial 
population  P 

(G);  while  stopping  criterion(s)  doesn’t  meet  do 


i.  Choose  individuals  to  create  mating  pool 
via  selection  operator. 

ii. Produce  new  individuals  via  crossover 

operator. 

iii.  Mutate  obtained  individuals  using  mutation 

operator. 

iv.  Evaluate  fitness  value  of  new  individuals. 

v.  Add  new  individuals  with  population. 

vi.  Sort  population  with  respect  to  fitness 
value. 

vii.  Choose  top  N  individuals  as  population  of 

next  generation, 
end  while 

Return  the  fittest  individual  as  the  solution. 

Every  individual  are  transformed  from  one  generation 
to  next  generation  depending  on  fitness  value  i.e 
individual  with  high  fitness  value  has  high  probability 
to  participate  in  upcoming  generation.  GA  employee 
crossover  operator  for  exploration  and  mutation 
operator  for  exploitation.  MDKP  uses  one  point 
crossover  and  random  mutation.  Number  of  crossover 
operations  are  govern  through  crossover  rate  while 
number  of  mutation  operations  are  govern  through 
mutation  rate.  A  cycle  of  GA  comprise  of  applying 
GA  operations  i.e.  evaluation,  reproduction,  crossover 
and  mutation.  Each  generation  produce  a  set  of 
individuals.  GA  termination  return  the  fittest 
individual  as  solution.  The  details  of  GA  operators  i.e 
selection,  crossover,  and  mutation  are  explained  in 

[14] . 

III.  RELATED  WORK 

J.P.  Martin  and  C.B  Neto  and  M.K.  Crocomo  made 
compar-ison  among  four  linkage  learning  based 
genetic  algorithm  for  multidimensional  knapsack 
problem  namely,  extended  corn-pact  genetic 
algorithm  (eCGA),  bayesian  optimization  algo-rithm 
(BOA)  with  detection  graphs,  BOA  with  community 
detection  and  linkage  tree  genetic  algorithm  (LTGA) 

[15] -  eCGA  and  LTGA  have  better  exploration  and 
exploitation  capabilities  than  BOA  versions  with 
small  size  population  (N=100)  and  produce  better 
quality  solutions  for  MDKPs.  All  algorithms  with 
large  population  size  produce  approxi-mately  similar 
quality  solutions  for  MDKPs  but  these  have  different 
running  time  and  function  evaluations.  When  con¬ 
sider  solutions  quality,  functions  evaluations  and 
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running  time  of  algorithms,  BOA  versions  have  better 
running  time  than  eCGA  and  LTGA.  It  is  very 
difficult  to  make  comparison  for  computational  time 
between  eCGA  and  LTGA  because  these  have 
different  reproduction  operators. 

Khuri,  Back  and  Heitkotter  proposed  a  GA  for  0/1 
MDKP  [16].  Algorithm  allowed  breeding  and 
participation  of  infeasi-ble  solutions  during  search.  In 
this  GA,  reproduction  proceeds  by  combining  the 
partial  information  from  all  element  of  population.  A 
test  suite  with  few  problems  were  used  to  test  this 
GA;  only  moderate  quality  solutions  were  produced. 
Hoff,  Lokentangen  and  Mittet  proposed  a  GA  with 
proper  tuned  parameters  and  search  mechanisms  in 
which  only  feasible  solutions  were  allowed  during 
search  [17].  This  GA  discovered  optimal  for  54 
problems  out  a  suite  of  55  problems. 

Rudolph  and  Sprave  proposed  a  GA  in  which 
selection  of  parent  is  not  unrestricted  as  in  standard 
GA  but  is  restricted  be-tween  neighboring  solutions 
[18].  In  their  GA  every  infeasible  solutions  were 
penalized  as  GA  of  Khursi,  Back  and  Heitkotter  [16]. 
The  premier  component  of  this  GA  be  solutions  local 
interaction  with  in  a  spatial  structured  population  and 
self  ad-justed  controlling  mechanism  of  selection 
pressure.  P.  C.  Chu  and  J.E.  Beasley  proposed 
heuristic  dependent  GA  for  MDKP  [19].  In  their  GA 
heuristic  operator  was  employed  which  work  with 
problem  specific  knowledge.  Heuristic  operator 
repaired  infeasible  solutions  and  algorithm  discover 
optimal  solution  among  feasible  solutions.  C.  Cota 
and  J.M.  Troya  proposed  another  GA  which  have  an 
improvement  mechanism  whose  objective  is  to 
convert  infeasible  solutions  in  search  space  to  feasible 
solutions  [20],  Experimental  results  were  better  to 
basic  GA,  but  not  as  good  as  Chu  and  Beasley’s  GA 
results. 

IV.  CLUSTERED  GENETIC  ALGORITHM  FOR 
MULTIDMENSIONAL  KNAPSACK  PROBLEM 


In  this  work,  clustered  genetic  algorithm  is  developed 
for  multidimensional  knapsack  problem.  CGA  is 
explained  in  algorithm  2.  CGA  uses  an  advanced 
selection  method  which  employee  fuzzy  and  roulette 
wheel  selection  called  fuzzy  roulette  wheel  selection. 
Selection  mechanism  always  choose  first  individual 
and  assigned  it  to  first  partition  of  roulette  wheel. 
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Partition  size  is  relative  to  fitness  value.  Next 
similarity  value  between  first  individual  and 
remaining  individuals  are  estimated.  The  individual 
with  highest  similarity  value  occupy  the  second 
partition  of  roulette  wheel.  This  procedure  repeat  until 
all  individuals  are  allocated  to  partition  in  roulette 
wheel.  Figure  1  show  the  4  individuals  in  binary 
representation  and  procedure  of  assigning  individual 
to  the  partition  of  roulette  wheel.  Then  mechanism 
determined  overlapping  area  between  adjacent 
individuals.  Overlapping  area  size  is  correlated  by 
similarity  value  between  adjacent  individuals.  In 
mating  pool  formation,  a  random  number  is 
generated.  After  for  each  individual  membership 
degree  to  which  generated  number  belongs  is 
calculated.  Individual  with  highest  membership 
degree  is  selected.  This  process  repeat  until  mating 
pool  size  is  equal  to  population  size.  Figure  2  show 
the  fuzzy  roulette  wheel  selection  mechanism  for  4 
individuals  in  Figure  1. 


Individual  1  I  I  |  0  I  0  I  I  I  I  I  0  I  I  I  T 
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Fig.  1 :  Procedure  of  assigning  individual  to  partition 
of  roulette  wheel . 


CGA  adopt  a  special  crossover  operator  which  uses 
the  hierarchical  algorithm  to  form  clusters. 
Individuals  in  the  mating  pool  are  divided  into  two 
clusters  using  hierarchical  clustering  algorithm.  In  the 
hierarchical  algorithm,  single  linkage  procedure  given 
in  eq  (2)  is  apply  to  estimate  similarity  between  the 
two  clusters. 


Fig.  2:  Fuzzy  roulette  wheel. 


S(X;  Y  )  =  M  AX  fs(x;  y)g 
x2X;y2Y 


(4) 
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Where  X  and  Y  are  two  clusters  and  s(x,y)  stand  for 
similarity  between  individuals  x  and  y.  A  pair  of 
offsprings  are  generated  by  mating  of  a  random  parent 
from  first  cluster  and  a  random  parent  from  second 
cluster.  One  point  crossover  operator  is  employed. 
This  step  is  continued  N/2-1  times.  Random  mutation 
is  performed  for  each  offspring  to  keep  diversity  and 
to  defeat  the  problem  of  premature  convergence. 


Cluster  1  Cluster  2 


Fig.  3:  Selection  of  parent  pair 

for  crossover. 

Individual  1  |  1  |  0  |  0  |  1  |  1 

o  |  i  |  i 

Individual  2  |  0  |  1  |  0  |  1  |  1 

1  1  *  1  ° 

Offspring  1  1  |  0  |  0  *  ^ 

Offspring  2  0  |  1  |  0  1  l 

Fig.  4:  One  point  cros 

'  Crossover  point 

1  1  1  1  1  0  1 

0  1  1  1  1  1 

isover. 

V.  EXPERIMENTAL  RESULTS  AND 
DISCUSSION 

To  show  CGA  is  better  technique  to  GA,  experiments 
has  carried  out  over  30  standard  problems  for 
multidimensional  knapsack. 

Algorithm  2  Clustered  Genetic  Algorithm  (CGA! 


Initialize  the  control  parameters  ; 

Initialize  randomly  N  individuals  as  initial 
population  P  (G);  while  stopping  criterion(s) 
doesn’t  meet  do 


i.  Choose  individuals  to  create  mating  pool  via 

fuzzy  roulette  wheel  selection  operator. 

ii.  Create  two  clusters  of  individuals  using 

hierarchical  clustering  algorithm . 

iii.  (iii)Produce  new  individuals  by  using 
crossover  operator  with  a  random  parent 
from  cluster  1  and  a  random  parent  from 
cluster  2. 

iv.  Mutate  obtained  individuals  using  mutation 

operator. 

v.  Evaluate  fitness  value  of  new  individuals. 

vi.  Add  new  individuals  with  population. 

vii.  Sort  population  with  respect  to  fitness  value. 

viii.  Choose  top  N  individuals  as  population  of 

next  generation. 

end  while 

Return  the  fittest  individual  as  the  solution. 

A.  Experimental  Environment 

GA  and  CGA  depict  in  previous  section  were  imple- 
mentd  for  multidimensional  Knapsack.  Experiments 
were  de-signed  to  estimate  average  mean  error  and 
average  mean  execution  time  for  GA  and  CGA.  Data 
set  for  multidimen- sional  knapsack  is  available  at 
http://people.brunel.ac.uk/  mas- 

tjjb/jeb/orlib/files/mknap2.txt.  Crossover  rate  CR  = 
0.7  and  mutation  rate  MR  =  0.01.  The  experiments 
were  conducted  on  7th  generation  intel  core  i-7-7700k 
processor  (4.2  GHz)  and  coded  in  C++  whose 
compilation  is  done  using  Dev  C++  compiler. 


Fig.  5:  Error  comparision  between  GA  and  CGA. 
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B.  Results  and  Analysis 

Experimental  results  of  GA  and  CGA  for  MDKPs  are 
given  in  Table  I  and  Table  II  respectively.  Column  1 
denote  instance  no  and  column  2  represent  MDKP, 
column  3  and  4  show  number  of  knapsacks  and 
number  of  objects  corresponding  to  problem,  columns 
5  denotes  the  optimal  value,  columns  6-10  keeps  the 
output  value  of  five  run,  simulation  statical 
information  is  given  in  columns  11-14  and  column  15 
show  mean  time  taken  in  execution.  GA  and  CGA  has 
average  mean  error  4.49%  and  1.764%  respectively 
while  GA  and  CGA  has  average  mean  execution  time 
2.34  seconds  and  2.34  seconds  respectively.  CGA  has 
considerable  less  average  mean  error  in  comparison  of 
GA  but  CGA  has  slightly  higher  average  mean 
execution  time  in  comparison  of  GA.  Figure  5  plot  the 
average  mean  error  against  the  problem  instance  and 
Figure  6  plot  the  average  mean  execution  time  against 
the  problem  instance. 


Fig.  6:  CPU  running  time  comparison  between  GA 
and  CGA. 


VI.  CONCLUSION  AND  FUTURE  WORK 

This  paper  addressed  multidimensional  knapsack 
problem  via  the  cluster  genetic  algorithm  to  find 
optimal.  CGA  employ  the  fuzzy  roulette  selection 
mechanism  for  individual  selection  to  form  mating 
pool.  CGA  adopts  a  special  kind  of  crossover  operator 
in  which  uses  hierarchical  clustering  algorithm  to 
form  two  clusters  from  mating  pool.  A  pair  of 
offspring  is  generated  by  mating  of  a  random  parent 
from  first  cluster  and  a  random  parent  from  second 
cluster.  CGA  performance  has  examined  against  GA 
w.r.t  30  standard  MDKPs  over  two  criterion  i.e 
average  mean  error  and  average  mean  execution  time. 


As  GA  and  CGA  experimental  results  are  shown, 

CGA  has  better  performance  than  GA  for  MDKP.  A 

comparative  study  of  different  mutation  operators 

with  CGA  will  be  studied  in  the  future  work. 
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TABLE  I:  Result  of  Genetic  Algorithm  For  Multidimensional  Knapsack  Problems. 
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35 
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3104 
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3104 

3109 

2.42 

2.12 

25 

PB1 

4 

27 

3090 

3012 

3001 

3017 

3005 

3015 

301 

7 

3001 

3010 

2.58 

1.93 
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4 

34 
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8 
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27 
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2 

29 
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8 
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8 

94105 
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PB5 

10 

20 
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5 
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29 
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30 

40 
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769 
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1009 

2.51 

3.49 

TABLE  II:  Result  of  Clustered  Genetic  Algorithm  for  Multidimensional  Knapsack  Problems. 
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